High-dimensional Neural Network Potentials Research Articles

Active and Transfer Learning of High-Dimensional Neural Network Potentials for Transition Metals.

Classical molecular dynamics (MD) simulations represent a very popular and powerful tool for materials modeling and design. The predictive power of MD hinges on the ability of the interatomic potential to capture the underlying physics and chemistry. There have been decades of seminal work on developing interatomic potentials, albeit with a focus predominantly on capturing the properties of bulk materials. Such physics-based models, while extensively deployed for predicting the dynamics and properties of nanoscale systems over the past two decades, tend to perform poorly in predicting nanoscale potential energy surfaces (PESs) when compared to high-fidelity first-principles calculations. These limitations stem from the lack of flexibility in such models, which rely on a predefined functional form. Machine learning (ML) models and approaches have emerged as a viable alternative to capture the diverse size-dependent cluster geometries, nanoscale dynamics, and the complex nanoscale PESs, without sacrificing the bulk properties. Here, we introduce an ML workflow that combines transfer and active learning strategies to develop high-dimensional neural networks (NNs) for capturing the cluster and bulk properties for several different transition metals with applications in catalysis, microelectronics, and energy storage, to name a few. Our NN first learns the bulk PES from the high-quality physics-based models in literature and subsequently augments this learning via retraining with a higher-fidelity first-principles training data set to concurrently capture both the nanoscale and bulk PES. Our workflow departs from status-quo in its ability to learn from a sparsely sampled data set that nonetheless covers a diverse range of cluster configurations from near-equilibrium to highly nonequilibrium as well as learning strategies that iteratively improve the fingerprinting depending on model fidelity. All the developed models are rigorously tested against an extensive first-principles data set of energies and forces of cluster configurations as well as several properties of bulk configurations for 10 different transition metals. Our approach is material agnostic and provides a methodology to transfer and build upon the learnings from decades of seminal work in molecular simulations on to a new generation of ML-trained potentials to accelerate materials discovery and design.

Review of High Dimensional Neural Network Potentials

Over the past few years, machine learning potentials (MLPs) have been fully developed and can now be applied to a variety of large-scale atomic simulations. MLPs encompass a number of different machine learning algorithms, of which neural network potentials are the most dominant and representative, and have achieved great success. This review focuses on the second generation of neural network potentials, high-dimensional neural network potentials (HDNNPs). While HDNNP has accomplished noteworthy results in the implementation of ion diffusion, nuclear magnetic resonance parameters, defect formation, and thermal conductivity, it still encounters hurdles in the compilation of training datasets, transferability, accuracy constraints, and the management of multiple chemical species. In spite of this, HDNNP remains a highly promising methodology. Looking forward, we anticipate the development of more specialized software for HDNNP to augment research efficiency and lower the barrier to usage.

From Oxygen Redox to Sulfur Redox: A Paradigm for Li-Rich Layered Cathodes.

The utilization of anionic redox chemistry provides an opportunity to further improve the energy density of Li-ion batteries, particularly for Li-rich layered oxides. However, oxygen-based hosts still suffer from unfavorable structural rearrangement, including the oxygen release and transition metal (TM)-ion migration, in association with the tenuous framework rooted in the ionicity of the TM-O bonding. An intrinsic solution, by using a sulfur-based host with strong TM-S covalency, is proposed here to buffer the lattice distortion upon the highly activating sulfur redox process, and it achieves howling success in stabilizing the host frameworks. Experimental results demonstrate the prolonged preservation of the layered sulfur lattice, especially the honeycomb superlattice, during the Li+ extraction/insertion process in contrast to the large structural degeneration in Li-rich oxides. Moreover, the Li-rich sulfide cathodes exhibited a negligible overpotential of 0.08 V and a voltage drop of 0.13 mV/cycle, while maintaining a substantial reversible capacity upon cycling. These superior electrochemical performances can be unambiguously ascribed to the much shorter trajectories of sulfur in comparison to those of oxygen revealed by molecular dynamics simulations at a large scale (∼30 nm) and a long time scale (∼300 ps) via high-dimensional neural network potentials during the delithiation process. Our findings highlight the importance of stabilizing host frameworks and establish general guidance for designing Li-rich cathodes with durable anionic redox chemistry.

Neural Network Approach for a Rapid Prediction of Metal-Supported Borophene Properties.

We developed a high-dimensional neural network potential (NNP) to describe the structural and energetic properties of borophene deposited on silver. This NNP has the accuracy of density functional theory (DFT) calculations while achieving computational speedups of several orders of magnitude, allowing the study of extensive structures that may reveal intriguing moiré patterns or surface corrugations. We describe an efficient approach to constructing the training data set using an iterative technique known as the "adaptive learning approach". The developed NNP is able to produce, with excellent agreement, the structure, energy, and forces obtained at the DFT level. Finally, the calculated stability of various borophene polymorphs, including those not initially included in the training data set, shows better stabilization for ν ∼ 0.1 hole density, and in particular for the allotrope α . The stability of borophene on the metal surface is shown to depend on its orientation, implying structural corrugation patterns that can be observed only from long-time simulations on extended systems. The NNP also demonstrates its ability to simulate vibrational densities of states and produce realistic structures with simulated STM images closely matching the experimental ones.

Diffusion and Coalescence of Phosphorene Monovacancies Studied Using High-Dimensional Neural Network Potentials.

The properties of two-dimensional materials are strongly affected by defects that are often present in considerable numbers. In this study, we investigate the diffusion and coalescence of monovacancies in phosphorene using molecular dynamics (MD) simulations accelerated by high-dimensional neural network potentials. Trained and validated with reference data obtained with density functional theory (DFT), such surrogate models provide the accuracy of DFT at a much lower cost, enabling simulations on time scales that far exceed those of first-principles MD. Our microsecond long simulations reveal that monovacancies are highly mobile and move predominantly in the zigzag rather than armchair direction, consistent with the energy barriers of the underlying hopping mechanisms. In further simulations, we find that monovacancies merge into energetically more stable and less mobile divacancies following different routes that may involve metastable intermediates.

High-Dimensional Neural Network Potentials for Accurate Prediction of Equation of State: A Case Study of Methane.

Machine learning-based interatomic potentials, such as those provided by neural networks, are increasingly important in molecular dynamics simulations. In the present work, we consider the applicability and robustness of machine learning molecular dynamics to predict the equation of state properties of methane by using high-dimensional neural network potentials (HDNNPs). We investigate two different strategies for generating training data: one strategy based upon bulk representations using periodic cells and another strategy based upon clusters of molecules. We assess the accuracy of the trained potentials by predicting the equilibrium mass density for a wide range of thermodynamic conditions to characterize the liquid phase, supercritical fluid, and gas phase, as well as the liquid-vapor coexistence curve. Our results show an excellent agreement with reference phase diagrams, with an average error below ∼2% for all studied phases. Moreover, we confirm the applicability of models trained on cluster data sets for producing accurate and reliable results.

How to train a neural network potential.

The introduction of modern Machine Learning Potentials (MLPs) has led to a paradigm change in the development of potential energy surfaces for atomistic simulations. By providing efficient access to energies and forces, they allow us to perform large-scale simulations of extended systems, which are not directly accessible by demanding first-principles methods. In these simulations, MLPs can reach the accuracy of electronic structure calculations, provided that they have been properly trained and validated using a suitable set of reference data. Due to their highly flexible functional form, the construction of MLPs has to be done with great care. In this Tutorial, we describe the necessary key steps for training reliable MLPs, from data generation via training to final validation. The procedure, which is illustrated for the example of a high-dimensional neural network potential, is general and applicable to many types of MLPs.

Anisotropic molecular coarse-graining by force and torque matching with neural networks.

We develop a machine-learning method for coarse-graining condensed-phase molecular systems using anisotropic particles. The method extends currently available high-dimensional neural network potentials by addressing molecular anisotropy. We demonstrate the flexibility of the method by parametrizing single-site coarse-grained models of a rigid small molecule (benzene) and a semi-flexible organic semiconductor (sexithiophene), attaining structural accuracy close to the all-atom models for both molecules at a considerably lower computational expense. The machine-learning method of constructing the coarse-grained potential is shown to be straightforward and sufficiently robust to capture anisotropic interactions and many-body effects. The method is validated through its ability to reproduce the structural properties of the small molecule's liquid phase and the phase transitions of the semi-flexible molecule over a wide temperature range.

Resolving the Perovskite Degradation Mechanism by Machine Learning Potential: The Case of CsPbI3

The perovskite prototype is one of the most promising solar cell materials. However, perovskite suffers from a phase transition leading to thermodynamic instability, which tightly influences the solar cell operation performance. Thus, modulating transition dynamics would extend its lifetime, which needs an in-depth understanding of the potential energy surface (PES) and the phase transition kinetics at the atomic level. In this work, taking CsPbI3 as an example of a perovskite prototype, we map out the PES and resolve the three lowest energy barrier paths of γ-CsPbI3 degradation by using a stochastic surface walking method integrated with high-dimensional neural-network potential. Path I is γ-CsPbI3 to hexagonal δ′-CsPbI3, a five-step transition with (110)Pv to (001)hex with the energy barrier 0.25 eV/f.u.; Path II is γ-CsPbI3 to cmcm-CsPbI3, a two-step transition with an over all energy barrier 0.22 eV/f.u. and (001)Pv//(110)cmcm + [010]Pv//[001]cmcm; Path III is γ-CsPbI3 to δ-CsPbI3, a one-step transition without forming an inherent interface, with the highest energy barrier 0.34 eV/f.u. Interestingly, We find that with the substitution of the A-site and/or B-site by other atoms, such as Bi and Te, the γ-CsPbI3 to δ-CsPbI3 transition could be extensively hindered. In this work, by resolving the potential energy surface, we not only reveal the degradation mechanism at the atomic level but also find a way to design perovskites with high and long-term stability.

Accelerating Non-Empirical Structure Determination of Ziegler–Natta Catalysts with a High-Dimensional Neural Network Potential

The determination of catalyst nanostructures with first-principles accuracy using genetic algorithms (GA) is very demanding due to the cubic scaling of the computational cost of density functional theory (DFT) calculations. Here, we demonstrate, for the case of Ziegler–Natta MgCl2/TiCl4 nanoplates, how this structure determination can be accelerated by employing a high-dimensional neural network potential (HDNNP) of essentially DFT accuracy. First, when building HDNNPs for MgCl2/TiCl4 clusters with computationally tractable sizes, we found that the structural diversity in the training set is crucial for obtaining HDNNPs reliably describing the large variety of structures generated by GA. The resulting HDNNPs dramatically accelerated the structure determination while yielding results consistent with DFT. Subsequently, we developed a multistep adaptive procedure to construct a HDNNP for MgCl2/TiCl4 clusters consistent in size and TiCl4 coverage with experiments where prior DFT results were scarcely collected. The structure determination and analyses underline the importance of system size and composition in order to predict some experimentally known facts such as the surface morphology and population of isospecific sites.

Accurate Fourth-Generation Machine Learning Potentials by Electrostatic Embedding.

In recent years, significant progress has been made in the development of machine learning potentials (MLPs) for atomistic simulations with applications in many fields from chemistry to materials science. While most current MLPs are based on environment-dependent atomic energies, the limitations of this locality approximation can be overcome, e.g., in fourth-generation MLPs, which incorporate long-range electrostatic interactions based on an equilibrated global charge distribution. Apart from the considered interactions, the quality of MLPs crucially depends on the information available about the system, i.e., the descriptors. In this work we show that including─in addition to structural information─the electrostatic potential arising from the charge distribution in the atomic environments significantly improves the quality and transferability of the potentials. Moreover, the extended descriptor allows current limitations of two- and three-body based feature vectors to be overcome regarding artificially degenerate atomic environments. The capabilities of such an electrostatically embedded fourth-generation high-dimensional neural network potential (ee4G-HDNNP), which is further augmented by pairwise interactions, are demonstrated for NaCl as a benchmark system. Employing a data set containing only neutral and negatively charged NaCl clusters, even small energy differences between different cluster geometries can be resolved, and the potential shows an impressive transferability to positively charged clusters as well as the melt.

Neural Network Accelerated Investigation of the Dynamic Structure-Performance Relations of Electrochemical CO2 Reduction over SnO x Surfaces.

Heterogeneous catalysts, especially metal oxides, play a curial role in improving energy conversion efficiency and production of valuable chemicals. However, the surface structure at the atomic level and the nature of active sites are still ambiguous due to the dynamism of surface structure and difficulty in structure characterization under electrochemical conditions. This paper describes a strategy of the multiscale simulation to investigate the SnO x reduction process and to build a structure-performance relation of SnO x for CO2 electroreduction. Employing high-dimensional neural network potential accelerated molecular dynamics and stochastic surface walking global optimization, coupled with density functional theory calculations, we propose that SnO2 reduction is accompanied by surface reconstruction and charge density redistribution of active sites. A regulatory factor, the net charge, is identified to predict the adsorption capability for key intermediates on active sites. Systematic electronic analyses reveal the origin of the interaction between the adsorbates and the active sites. These findings uncover the quantitative correlation between electronic structure properties and the catalytic performance of SnO x so that Sn sites with moderate charge could achieve the optimally catalytic performance of the CO2 electroreduction to formate.

Machine learning transferable atomic forces for large systems from underconverged molecular fragments.

Machine learning potentials (MLP) enable atomistic simulations with first-principles accuracy at a small fraction of the costs of electronic structure calculations. Most modern MLPs rely on constructing the potential energy, or a major part of it, as a sum of atomic energies, which are given as a function of the local chemical environments up to a cutoff radius. Since analytic forces are readily available, nowadays it is common practice to make use of both, reference energies and forces, for training these MLPs. This can be computationally demanding since often large systems are required to obtain structurally converged reference forces experienced by atoms in realistic condensed phase environments. In this work we show how density-functional theory calculations of molecular fragments, which are too small to provide such structurally converged forces, can be used to learn forces exhibiting excellent transferability to extended systems. The general procedure and the accuracy of the method are illustrated for metal-organic frameworks using second-generation high-dimensional neural network potentials.

A high-dimensional neural network potential for molecular dynamics simulations of condensed phase nickel and phase transitions

ABSTRACT A high-dimensional neural network interatomic potential was developed and used in molecular dynamics simulations of condensed phase Ni and Ni systems with liquid–solid phase coexistence. The reference data set was generated by sampling the potential energy surface over a broad temperature-pressure domain using ab initio MD simulations to train a unified potential. Excellent agreement was achieved between bulk face-centred cubic nickel thermal expansion simulations and relevant experimental data. The same potential also yields accurate structures and diffusivities in the liquid state. The phase transition between liquid and solid phases was simulated using the two-phase interface method. The predicted melting point temperature is within a few kelvins of the literature value. The general methodology could be applied to describe crystals with much more complex phase behaviours.

Coupled Cluster Molecular Dynamics of Condensed Phase Systems Enabled by Machine Learning Potentials: Liquid Water Benchmark

Coupled cluster theory is a general and systematic electronic structure method, but in particular the highly accurate "gold standard" coupled cluster singles, doubles and perturbative triples, CCSD(T), can only be applied to small systems. To overcome this limitation, we introduce a framework to transfer CCSD(T) accuracy of finite molecular clusters to extended condensed phase systems using a high-dimensional neural network potential. This approach, which is automated, allows one to perform high-quality coupled cluster molecular dynamics, CCMD, as we demonstrate for liquid water including nuclear quantum effects. The machine learning strategy is very efficient, generic, can be systematically improved, and is applicable to a variety of complex systems.

Neural network interaction potentials for para-hydrogen with flexible molecules.

The study of molecular impurities in para-hydrogen (pH2) clusters is key to push forward our understanding of intra- and intermolecular interactions, including their impact on the superfluid response of this bosonic quantum solvent. This includes tagging with only one or very few pH2, the microsolvation regime for intermediate particle numbers, and matrix isolation with many solvent molecules. However, the fundamental coupling between the bosonic pH2 environment and the (ro-)vibrational motion of molecular impurities remains poorly understood. Quantum simulations can, in principle, provide the necessary atomistic insight, but they require very accurate descriptions of the involved interactions. Here, we present a data-driven approach for the generation of impurity⋯pH2 interaction potentials based on machine learning techniques, which retain the full flexibility of the dopant species. We employ the well-established adiabatic hindered rotor (AHR) averaging technique to include the impact of the nuclear spin statistics on the symmetry-allowed rotational quantum numbers of pH2. Embedding this averaging procedure within the high-dimensional neural network potential (NNP) framework enables the generation of highly accurate AHR-averaged NNPs at coupled cluster accuracy, namely, explicitly correlated coupled cluster single, double, and scaled perturbative triples, CCSD(T*)-F12a/aVTZcp, in an automated manner. We apply this methodology to the water and protonated water molecules as representative cases for quasi-rigid and highly flexible molecules, respectively, and obtain AHR-averaged NNPs that reliably describe the corresponding H2O⋯pH2 and H3O+⋯pH2 interactions. Using path integral simulations, we show for the hydronium cation, H3O+, that umbrella-like tunneling inversion has a strong impact on the first and second pH2 microsolvation shells. The automated and data-driven nature of our protocol opens the door to the study of bosonic pH2 quantum solvation for a wide range of embedded impurities.

Extending ab initio simulations for the ion-ion structure factor of warm dense aluminum to the hydrodynamic limit using neural network potentials

We calculate the intermediate scattering function of warm dense aluminum by using density functional theory molecular dynamics simulations. From this data set, we derive the static and dynamic ion-ion structure factors. By applying a generalized collective modes model, we can fit the excitation spectra of the ion system and thereby extract the dispersion for the ion acoustic modes, as well as the decay coefficients for the diffusive and collective modes. The results are discussed and compared with experimental data if available. We show that computational limitations prevent sufficient access to the hydrodynamic limit and demonstrate that this can be circumvented using high-dimensional neural network potentials. We extract the ionic thermal conductivity of aluminum in the hydrodynamic limit and compare to values computed using a Green-Kubo relation. We highlight the importance of partitioning the heat capacity into electronic and ionic contributions and only using the ionic contribution to compute the thermal conductivity of the ions in the hydrodynamic limit.

Transferable Neural Network Potential Energy Surfaces for Closed-Shell Organic Molecules: Extension to Ions.

Transferable high dimensional neural network potentials (HDNNPs) have shown great promise as an avenue to increase the accuracy and domain of applicability of existing atomistic force fields for organic systems relevant to life science. We have previously reported such a potential (Schrödinger-ANI) that has broad coverage of druglike molecules. We extend that work here to cover ionic and zwitterionic druglike molecules expected to be relevant to drug discovery research activities. We report a novel HDNNP architecture, which we call QRNN, that predicts atomic charges and uses these charges as descriptors in an energy model that delivers conformational energies within chemical accuracy when measured against the reference theory it is trained to. Further, we find that delta learning based on a semiempirical level of theory approximately halves the errors. We test the models on torsion energy profiles, relative conformational energies, geometric parameters, and relative tautomer errors.

High-dimensional neural network potentials for accurate vibrational frequencies: the formic acid dimer benchmark.

In recent years, machine learning potentials (MLP) for atomistic simulations have attracted a lot of attention in chemistry and materials science. Many new approaches have been developed with the primary aim to transfer the accuracy of electronic structure calculations to large condensed systems containing thousands of atoms. In spite of these advances, the reliability of modern MLPs in reproducing the subtle details of the multi-dimensional potential-energy surface is still difficult to assess for such systems. On the other hand, moderately sized systems enabling the application of tools for thorough and systematic quality-control are nowadays rarely investigated. In this work we use benchmark-quality harmonic and anharmonic vibrational frequencies as a sensitive probe for the validation of high-dimensional neural network potentials. For the case of the formic acid dimer, a frequently studied model system for which stringent spectroscopic data became recently available, we show that high-quality frequencies can be obtained from state-of-the-art calculations in excellent agreement with coupled cluster theory and experimental data.

Insights into lithium manganese oxide-water interfaces using machine learning potentials.

Unraveling the atomistic and the electronic structure of solid-liquid interfaces is the key to the designof new materials for many important applications, from heterogeneous catalysis to battery technology. Density functional theory (DFT) calculations can, in principle, provide a reliable description of such interfaces, but the high computational costs severely restrict the accessible time and length scales. Here, we report machine learning-driven simulations of various interfaces between water and lithium manganese oxide (LixMn2O4), an important electrode material in lithium ion batteries and a catalyst for the oxygen evolution reaction. We employ a high-dimensional neural network potential to compute the energies and forces several orders of magnitude faster than DFT without loss in accuracy. In addition, a high-dimensional neural network for spin prediction is utilized to analyze the electronic structure of the manganese ions. Combining these methods, a series of interfaces is investigated by large-scale molecular dynamics. The simulations allow us to gain insights into a variety of properties, such as the dissociation of water molecules, proton transfer processes, and hydrogen bonds, as well as the geometric and electronic structure of the solid surfaces, including the manganese oxidation state distribution, Jahn-Teller distortions, and electron hopping.